Method and apparatus for temporal wavelet compression

ABSTRACT

A system is described that comprises a method and apparatus for further compressing an already compressed data stream, using a mathematical comparison computation. The method comprises: receiving a data stream that has been compressed using a first compression protocol; decompressing the data stream using the first compression protocol; reducing temporal redundancy in the data stream using a comparison protocol; and recompressing the data stream using the first compression protocol. The apparatus comprises: a decompression unit to receive and decompress a compressed data stream to produce decompressed data; a comparison unit, coupled to the decompression unit, to perform a mathematical comparison operation on the decompressed data to produce resultant data; and a recompression unit to compress the resultant data to produce a further compressed data stream.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This patent application is a continuation of U.S. patent applicationSer. No. 10/096,490 filed on Mar. 13, 2002; which is related to andclaims priority to U.S. provisional application entitled TEMPORALWAVELET COMPRESSION ALGORITHM having Ser. No. 60/275,113, by Joseph P.Vidunas, filed Mar. 13, 2001; and which are both hereby incorporated byreference into this application.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention is directed to data compression and, moreparticularly to further compression of already compressed video.

2. Description of the Related Art

Image (e.g., video) data is often compressed or processed by usingwavelet analysis. Wavelets are mathematical functions that can segment adigital signal into different frequency bands. Wavelets have someadvantages over Fourier transforms, but are roughly analogous. Knowncompression systems convert raw image data into wavelet transform data,then run-length encode or Hoffman encode the data.

Lossy compression provides high degrees of compression and results invery small compressed files, but there is a certain amount of loss whenthey are restored (cf. lossless compression, which does not provide ashigh a degree of compression, but which does not result in any loss uponrestoration). Wavelet compression is a lossy compression method used forcolor images and video. Instead of compressing small blocks of 8×8pixels as in JPEG and MPEG compression methods, the wavelet algorithmscompress the entire image with ratios of up to 300:1 for color and 50:1for gray scale. An example of a codec that performs wavelet compressionis the ADV601 from Analog Devices, Inc. of Norwood, Mass.

Another known way to compress video is by reducing temporal redundancyusing an exclusive-OR (XOR) comparison, an example of which isillustrated in FIG. 1. In FIG. 1, previous frame 101 and current frame102 are individual, successive frames within a video stream. Each cellof the matrices that form frames 101 and 102 in FIG. 1 (e.g., cells 104to 111) represents a wavelet coefficient. Upon performing an XORoperation on frames 101 and 102, resultant matrix 103 is produced, whichcontains cells (e.g., 112-115) corresponding to the cells in the sameposition in frames 101 and 102, the contents of which are the result ofan XOR operation on the corresponding cells in frames 101 and 102. Forexample, an XOR operation is performed on the contents of cells 104 and108 which have a value of ‘0’ to produce a ‘0’ result in cell 112. Thesame result is produced in cells 113-115 when an XOR operation isperformed on cells 105 and 109, 106 and 110, and 107 and 111,respectively, because the contents of the corresponding pairs of cellsin frames 101 and 102 are identical. Upon performing an XOR operation oncells of frames 101 and 102 with differing contents, such as cells 116and 117, which contain a ‘0’ and ‘1,’ respectively, a ‘1’ results in box118 of matrix 103.

XOR comparison is useful in video streams where each video frame imageis usually the same (e.g., security cameras). XOR comparison can be usedto dramatically reduce the size of the compressed file because each boxof each frame is usually identical to the same box in a subsequentframe, and can therefore be compressed accordingly. However, it is notpossible to perform XOR comparison after known wavelet compression of avideo stream, like that performed by the ADV601, without reversing therun-length and Huffman encoding to obtain access to the waveletcoefficients.

SUMMARY OF THE INVENTION

It is an aspect of the present invention to provide a method andapparatus to perform a mathematical comparison operation on the waveletdata from sequential frames of a data stream at the quantizedcoefficient layer across each of the wavelet frequency bands. It isanother aspect of the present invention to selectively apply the resultsof a mathematical comparison operation where the gain thereby isdetermined to be significant.

The above aspects can be attained by a method for further compressing analready compressed data stream, using a mathematical comparisoncomputation by receiving a data stream that has been compressed using afirst compression protocol; decompressing the data stream using thefirst compression protocol; reducing temporal redundancy in the datastream using a comparison protocol; and recompressing the data streamusing the first compression protocol.

The method can be performed by an apparatus having a decompression unitto receive and decompress a compressed data stream to producedecompressed data; a comparison unit, coupled to the decompression unit,to perform a mathematical exclusive-OR operation on the decompresseddata to produce resultant data; and a recompression unit to compress theresultant data to produce a further compressed data stream.

The decompression and recompression units may share components. Forexample, the decompression and recompression units may include anentropy (e.g. Huffman) encoding/decoding unit, to receive the compresseddata stream and perform decompression to produce decompressed data and arun-length encoding/decoding unit, coupled with the entropyencoding/decoding unit to decode the decompressed data into quantizeddata that is supplied to the comparison unit. The run-lengthencoding/decoding unit then compresses the resultant data from thecomparison unit to produce run-length encoded data, and the entropyencoding/decoding unit compresses the run-length encoded data to producethe further compressed data stream.

The aspects described above, together with other aspects and advantagesthat will be subsequently apparent, reside in the details ofconstruction and operation as more fully hereinafter described andclaimed, reference being made to the accompanying drawings forming apart hereof, wherein like numerals refer to like parts throughout.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an example of a conventional exclusive-OR comparisonoperation.

FIG. 2 is a flowchart of a method for temporal wavelet compression,according to embodiments of the present invention.

FIG. 3 is a block diagram of an apparatus for temporal waveletcompression, according to embodiments of the present invention.

FIG. 4 is a flowchart of a method for temporal wavelet compression aftera data stream has already been compressed by an ASIC for waveletcompression, according to embodiments of the present invention.

FIG. 5 is a block diagram of an apparatus for temporal waveformcompression after a data stream has already been compressed by an ASICfor wavelet compression, according to embodiments of the presentinvention.

FIG. 6 is a block diagram of a video system with a temporal waveletcompression unit, according to embodiments of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Temporal wavelet compression reduces the data rate of an intra-framewavelet video stream by applying a mathematical exclusive-OR (XOR)operation on the wavelet data from sequential frames at the quantizedwavelet coefficient layer across each of the wavelet frequency bands.This means the XOR operation is applied to all of the wavelet bands. Thenumber of wavelet bands can vary in number depending on the intra-framewavelet component. For example, an off-the-shelf compression ASIC, suchas Analog Devices ADV601, used in an embodiment of the present invention(see the discussion of FIGS. 4 and 5, below) produces 42 waveletfrequency bands. There is no selection of particular wavelet bands,unless a frequency band threshold is used as described below (in regardto FIG. 4).

FIG. 2 is a flowchart of a method for temporal wavelet compression,according to embodiments of the present invention. Specifically, FIG. 2illustrates method 130, which begins where quantized data is received132. The result of the quantization in the intra-frame wavelet processis a matrix of wavelet coefficients rounded to distinct values eitherpreconfigured or calculated by the intra-frame wavelet process. An XORoperation is then performed 134 on the quantized data. A last framebuffer simply holds 136 the wavelet coefficients from the last frame toprovide a basis for an XOR comparison. The equation for the comparisonis:

q _(n) [x,y,b]=q _(n) [x,y,b]⊕+q _((n−)1)[x,y,b]

where n is the frame number and q is a quantized wavelet coefficient ata point determined by pixel locations x and y in the wavelet bandcorresponding to value b. The XOR operation was chosen for thecomparison for the following reasons: (1) it results in a zero value ifthe coefficients in sequential frames are the same; (2) it will notoverflow the double byte capacity of the wavelet coefficients; and (3)it is easily reversed in the decode operation by applying the exact sameequation and (4) XOR can be calculated by Pentium II class processors inonly one clock cycle.

Application of temporal wavelet compression produces a temporal waveletframe in which the quantized value at a particular band, horizontal, andvertical position is equal to the result of an XOR operation betweenitself and the corresponding value in the previous frame. Since thequantized wavelet coefficients are very similar from frame to frame inthe average security video scene, the mathematical XOR operation willresult in a zero value in more than 98% of the operations. Conventionalrun-length encoding is then applied 138 to these primarily zero valuesto produce a significant data reduction from the original frame.Further, the data is entropy encoded 140.

FIG. 3 is a block diagram of an apparatus for temporal waveletcompression, according to embodiments of the present invention.Specifically, FIG. 3 illustrates XOR comparison unit 150, which iscoupled to receive quantized data. In an embodiment, XOR comparison unit150 performs the XOR operation described above. Last frame buffer 152holds the wavelet coefficients from the last frame to provide a basisfor an XOR comparison. Run-length encoding unit 154 is coupled with XORcompression unit 150, and encodes the XOR-compressed data. Entropyencoding unit 156 is coupled with run-length encoding unit 154, andencodes the run-length encoded data. In an embodiment, entropy encodingunit 156 is a Huffman encoding unit. The run-length and entropy encodingcan be performed according to conventional protocols.

FIG. 4 is a flowchart of a method for temporal wavelet compression aftera data stream has already been compressed by an ASIC for waveletcompression, according to embodiments of the present invention.Specifically, FIG. 4 illustrates method 170, which begins where acompressed data stream is received 172. In an embodiment, the compresseddata stream is received from a compression application-specificintegrated circuit (compression ASIC) (e.g., an ADV601), and is acompressed video stream from a security camera. In this embodiment, thevideo stream is compressed using run-length encoding and entropyencoding. The data stream is then decompressed 173, which, in theembodiment just described, involves reversing the run-length and entropyencoding. A check is made to determine whether a key frame is thecurrent frame received 174. If so, the original frame is copied to anout buffer 175, the wavelet data is stored to a last frame buffer 176,and the out buffer is transmitted 177. If not, a check with outsideconditions is made to determine whether to recalculate the frequencyband compression threshold 178 to determine at what point waveletfrequency bands should be skipped, where the skipped frequency bandshave already been compressed enough using the first compressionprotocol. A threshold is either recalculated 179 or left at is existingvalue.

A frequency band is selected 180, and then is checked to determinewhether the current frequency band is within the threshold 181. If not,the original data is copied to the out buffer 183 (and the methodcontinues as indicated in FIG. 4). If so, a XOR operation is performedusing the wavelet coefficient data for the current frequency band fromthe current frame and the previous frame 182. The frequency band is thenrecompressed 184 using the same compression protocol as was used whenreceived, e.g., run-time and entropy encoding. Raw data afterquantization is generally zero in 60% to 80% of the coefficients. Afterthe additional inter-frame XOR comparison, the percentage increases tobetween 90% and 99%.

A check is then made to determine whether the XOR-resultant data isfurther compressed over the received data stream 185. If so, theXOR-resultant data is copied to the out buffer 186. If not, the originaldata is copied to the out buffer 183.

A check is made to determine whether all frequency bands are complete187. If not, another band is selected 180 (and the method continues asindicated in FIG. 4). If so, the wavelet data is stored to a last framebuffer 176, and the out buffer is transmitted 177. In an embodiment, theresult of method 170 is transmitted. In an embodiment, the result ofmethod 170 is stored.

As noted above, off-the-shelf components, which may include compressionASIC such as an ADV601, are used in the present implementation toperform the conventional intra-frame wavelet compression. Since thesecomponents completely encode the data, the run-length and entropyencoding are reversed to obtain quantized data for temporal waveletcompression. The present invention takes advantage of knowing theresults of the conventional intra-frame wavelet compression to make adetermination of how well each wavelet band has been compressed in therun-length and entropy encoding steps. A ratio of the size of theencoded data to the raw quantized data is compared to a frequency bandcompression threshold to determine if the particular band is already sosmall that it would be a waste to attempt to compress it any further.The threshold value can be scaled between a useful range of about 1-30%to maximize compression or minimize processor usage.

The frequency band compression threshold parameter allows the algorithmto be applied selectively to the bands where the highest data rates arefound. This allows scaling the compression to minimize data rate orprocessor utilization depending on the operational environment.

Applying the Temporal Wavelet Algorithm to the intra-frame waveletstream produces a greatly reduced data rate. In a test of average sampledata, this reduction was on the order of 40-60%. The mathematical XORcomparison is very simple and is relatively easy to accomplish usingonly software and existing processors. Using highly optimized softwarecode, this algorithm can be applied to QCIF wavelet frames at an averagerate over 100 frames per second on Pentium II class processors. Thefrequency band compression threshold parameter allows the algorithm tobe applied selectively to the bands where the highest data rates arefound. This allows scaling the compression to minimize data rate orprocessor utilization depending on the operational environment.

Alternative embodiments include using an intra-frame wavelet stream withno temporal data reduction, using some other temporal reductioninvolving perhaps motion estimation, or using another common temporalcompression scheme like the MPEG varieties.

FIG. 5 is a block diagram of an apparatus for temporal waveletcompression after a data stream has already been compressed by an analogdevice for temporal wavelet compression, according to an embodiment ofthe present invention. Specifically, FIG. 5 illustrates temporal waveletcompression unit 204 (illustrated using dashed lines), which comprisesentropy encoding/decoding unit 190, run-length encoding/decoding unit192, exclusive-OR compression unit 194, and last frame buffer unit 196.Entropy encoding/decoding unit 190 is coupled to unillustratedcomponents to send and receive compressed data, and encodes run-lengthencoded data or decodes entropy encoded data. Run-lengthencoding/decoding unit 192 is coupled with entropy encoding/decodingunit 190, and encodes XOR compared data or decodes run-length encodeddata. Exclusive-OR comparison unit 194 is coupled with run-lengthencoding/decoding unit 192, and performs XOR comparison on a datastream. Last frame buffer unit 196 holds the wavelet coefficients forthe previously processed frame to provide a basis for the XORcomparison. The apparatus illustrated in FIG. 5 is capable of furthercompressing already compressed data received from a compression ASIC byreducing temporally redundant data from the data stream. In anembodiment, entropy encoding/decoding unit 190 is a Huffmanencoder/decoder.

Embodiments of the present invention may also be incorporated into asystem. FIG. 6 is a block diagram of a video system with a temporalwaveform compression unit, according to embodiments of the presentinvention. Specifically, FIG. 6 illustrates system 199, in whichcomprises video camera 200 is connected to compression ASIC 202;compression ASIC 202 is connected to temporal wavelet compression unit204 (in an embodiment, via memory unit 205 as described below); temporalwavelet compression unit 204 is connected to storage unit 206 (in anembodiment, via memory unit 205 as described below); storage unit 206 isconnected to transmittal unit 208 (in an embodiment, via memory unit 205as described below); and transmittal unit 208 is connected to a network(not shown) for transmission of compressed data to network customers212.

In an embodiment, video camera 200 is a security camera. In anembodiment, compression ASIC 202 is an ADV601 as previously describedabove. In an embodiment, storage unit 206 is an online IDE hard drive.In an embodiment, transmittal unit 208 is a network interface. In anembodiment, system 199 also comprises memory unit 205, which can be PCSDRAM chips. In an embodiment, system 199 also comprises digitalrecorder 210 (illustrated in FIG. 6 using dashed lines), which comprisescompression ASIC 202, temporal wavelet compression unit 204, storageunit 206, transmittal unit 208, and memory unit 205 (if included insystem 199).

In an embodiment, system 199 is capable of generating a video signal,compressing it using entropy and run-length encoding, decompressing itusing the same protocols, performing an XOR operation to compress thedata, recompressing the XOR-compared data using entropy and run-lengthprotocols, and storing or transmitting the compressed data. The XOR andentropy/run-length encoded data stream is smaller than without the XORcomparison.

The chipset included in the system includes memories, processors, and/orASICs. Such memory includes a machine-readable medium on which is storeda set of instructions (i.e., software) embodying any one, or all, of themethodologies described herein. Software can reside, completely or atleast partially, within this memory and/or within the processor and/orASICs. For the purposes of this specification, the term“machine-readable medium” shall be taken to include any mechanism thatprovides (i.e., stores and/or transmits) information in a form readableby a machine (e.g., a computer). For example, machine-readable mediainclude read only memory (ROM), random access memory (RAM), magneticdisk storage media; optical storage media, flash memory devices,electrical, optical, acoustical, or other form of propagated signals(e.g., carrier waves, infrared signals, digital signals, etc.), etc.

The many features and advantages of the invention are apparent from thedetailed specification and, thus, it is intended by the appended claimsto cover all such features and advantages of the invention that fallwithin the true spirit and scope of the invention. Further, sincenumerous modifications and changes will readily occur to those skilledin the art, it is not desired to limit the invention to the exactconstruction and operation illustrated and described, and accordinglyall suitable modifications and equivalents may be resorted to, fallingwithin the scope of the invention.

1. A method for further compressing a compressed data stream,comprising: receiving a data stream that has been compressed using afirst compression protocol and a second compression protocol;decompressing the data stream using the first compression protocol;reducing temporal redundancy in the data stream using a comparisonprotocol; and recompressing the data stream using the first compressionprotocol.
 2. The method as recited in claim 1, wherein the firstcompression protocol includes run-length encoding and entropy encodingof quantized data.
 3. The method as recited in claim 2, wherein theentropy encoding is Huffman encoding.
 4. The method as recited in claim1, wherein the second compression protocol includes wavelet compression.5. The method as recited in claim 1, further comprising: determiningwhether a recompressed data stream resulting from said recompressing isfurther compressed than the received data stream; and discarding therecompressed data stream if it is not further compressed than thereceived data stream.
 6. A computer-readable medium storingcomputer-executable instructions to perform a method comprising:receiving a data stream that has been compressed using a firstcompression protocol and a second compression protocol; decompressingthe data stream using the first compression protocol; reducing temporalredundancy in the data stream using a comparison protocol; andrecompressing the data stream using the first compression protocol. 7.The computer-readable medium as recited in claim 6, wherein the firstcompression protocol includes run-length encoding and entropy encodingof quantized data.
 8. The computer-readable medium as recited in claim7, wherein the entropy encoding is Huffman encoding.
 9. Thecomputer-readable medium as recited in claim 6, wherein the secondcompression protocol includes wavelet compression.
 10. Thecomputer-readable medium as recited in claim 6, further comprising:determining whether a recompressed data stream resulting from saidrecompressing is further compressed than the received data stream; anddiscarding the recompressed data stream if it is not further compressedthan the received data stream.
 11. An apparatus to further compress acompressed data stream, comprising: means for receiving a data streamthat has been compressed using a first compression protocol and a secondcompression method; means for decompressing the data stream using thefirst compression protocol; means for reducing temporal redundancy inthe data stream using a comparison protocol; and means for recompressingthe data stream using the first compression protocol.
 12. The apparatusas recited in claim 11, wherein the first compression protocol includesa run-length encoding and an entropy encoding of quantized data.
 13. Theapparatus as recited in claim 12, wherein the entropy encoding isHuffman encoding.
 14. The apparatus as recited in claim 11, wherein thesecond compression protocol includes wavelet compression.
 15. Theapparatus as recited in claim 11, further comprising: means fordetermining whether a recompressed data stream resulting from saidrecompressing is further compressed than the received data stream; andmeans for discarding the recompressed data stream if it is not furthercompressed than the received data stream.
 16. An apparatus to furthercompress a compressed data stream, comprising: a decompression unit toreceive and partially decompress the compressed data stream to producepartially decompressed data; a comparison unit, coupled to thedecompression unit, to perform a mathematical compare operation on thedecompressed data to produce resultant data; and a recompression unit tocompress the resultant data to produce a further compressed data stream.17. The apparatus as recited in claim 16, wherein said decompressionunit comprises: an entropy encoding/decoding unit, to receive thecompressed data stream and to decode the compressed data stream toproduce decoded data; and a run-length encoding/decoding unit, coupledto said entropy encoding/decoding unit and said comparison unit, tofurther decode the decoded data into quantized data supplied to saidcomparison unit as the partially decompressed data.
 18. The apparatus asrecited in claim 17, wherein the entropy encoding/decoding unit is aHuffman entropy encoder/decoder.
 19. The apparatus as recited in claim17, wherein said recompression unit shares said entropyencoding/decoding and run-length encoding/decoding units with saiddecompression unit by said run-length encoding/decoding unit encodingthe resultant data after said comparison unit processes the quantizeddata, to produce encoded data, and said entropy encoding/decoding unitcompressing the encoded data to produce the further compressed data. 20.The apparatus as recited in claim 16, wherein said comparison unitperforms the comparison operation by calculatingqn[x,y,b]=qn[x,y,b]⊕q(n−1)[x,y,b], where n is the frame number, andwhere q is a quantized wavelet coefficient at a point determined bypixel locations x and y in the wavelet band corresponding to value b.